-
Notifications
You must be signed in to change notification settings - Fork 5.5k
Fix no response / null response from LLM #3591
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Fix no response / null response from LLM #3591
Conversation
|
Hey @lucasgomide, do you think this could be possible solution for this? I got some feedback from folks who tried the feature branch |
|
Gentle ping on this one @lucasgomide |
|
Added to my list to review later today |
3807072 to
5c9ac8f
Compare
|
mind you checking the CI errors? |
All fixed now. |
| bool: True if the exception is due to context length exceeding | ||
| """ | ||
| if isinstance(exception, ValueError) and "None or empty" in str(exception): | ||
| return True |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: Early return bypasses context length validation logic
The function is_null_response_because_context_length_exceeded has an early return that returns True for any ValueError with "None or empty" in the message, without checking if the context length was actually exceeded. This contradicts the function's purpose and the accompanying tests. For example, the test for empty messages expects False, and the test for messages smaller than the context window also expects False, but both will incorrectly return True due to this early return. The early return at lines 441-442 should be removed so only the final condition (which correctly validates all criteria) is used.
This is a try solution for None / Empty response from LLM.
This PR assumes that this might be because of context length exceeded at the server side, which is not returning an error, just an empty response.
This PR captures the ValueError which is being raised and try again with summarization.
Note
Detect and handle null/empty LLM responses as context-window overflows, triggering summarization; update typings to
BaseAgentand add focused tests.is_null_response_because_context_length_exceededand route tohandle_context_lengthinCrewAgentExecutorandLiteAgent.LLMMessagetyping where needed.is_null_response_because_context_length_exceeded(exception, messages, llm)inutilities/agent_utils.py.is_context_length_exceededdocstring/signature; no behavioral regression.BaseAgentinTaskEvaluatorand update related imports/usages (e.g., tool utils typing).Written by Cursor Bugbot for commit da3f195. This will update automatically on new commits. Configure here.